Electronic apparatus and controlling method thereof

ABSTRACT

An electronic apparatus is disclosed. The electronic apparatus includes: a memory storing a first pattern image and a second pattern image, a communication interface comprising communication circuitry configured to communicate with an external terminal apparatus, a projection part including a projection lens, and a processor configured to: control the projection part to project the first pattern image on a screen member comprising a reflector located on a projection surface, and based on receiving a first photographed image which photographed the screen member from the external terminal apparatus through the communication interface, acquire transformation information based on the first photographed image and the first pattern image, control the projection part to project the second pattern image on the projection surface, and based on receiving a second photographed image which photographed the projection surface from the external terminal apparatus through the communication interface, perform color calibration based on the characteristic of the projection surface based on the second photographed image, the second pattern image, and the transformation information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/KR2022/000018 designating the United States, filed on Jan. 3, 2022,in the Korean Intellectual Property Receiving Office and claimingpriority to Korean Patent Application No. 10-2021-0052220, filed on Apr.22, 2021, in the Korean Intellectual Property Office and Korean PatentApplication No. 10-2021-0144246, filed on Oct. 27, 2021, the disclosuresof all of which are incorporated by reference herein in theirentireties.

BACKGROUND Field

The disclosure relates to an electronic apparatus including a projectionpart and a controlling method thereof, and for example, to an electronicapparatus that performs a color calibrating operation related to imageprojection, and a controlling method thereof.

Description of Related Art

In the case of an electronic apparatus projecting an image through aprojection part (e.g., a projector), a specific surface such as a wallor a ceiling may be a screen. In the case of a screen which is not adedicated screen, the color of the screen on which an image is projectedmay not be a standard white color. Accordingly, if the color of a screenis not a standard white color, there is a problem that the color of theoriginal image is not displayed as it is. For example, it is assumedthat the color of a screen is gray. In case a white image is projectedon a gray projection surface, there is a possibility that a user mayrecognize the image as a light gray image, but not a white image.

Even if the color of a screen is white from a user's point of view, thecolor may not perfectly coincide with the standard white color. In thecase of not using a dedicated screen, a problem that a user recognizesan image having a different color from the color of the original imagemay occur.

Even if a dedicated screen is used, a possibility that the color of thededicated screen may be discolored as time passes may exist.Accordingly, a problem that a user recognizes an image of a differentcolor from the color of the original image projected on the discoloredprojection surface may occur.

SUMMARY

Embodiments of the disclosure provide an electronic apparatus thatoutputs a pattern image to a screen member and acquires color spacetransformation information, and outputs the pattern image to aprojection surface and performs color calibration appropriate for theprojection surface, and a controlling method thereof.

An electronic apparatus according to an example embodiment of thedisclosure includes: a memory storing a first pattern image and a secondpattern image, a communication interface comprising communicationcircuitry configured to communicate with an external terminal apparatus,a projection part comprising a projector or image projecting lens, and aprocessor configured to: control the projection part to project thefirst pattern image on a screen member comprising a reflector located ona projection surface, and based on receiving a first photographed imagewhich photographed the screen member from the external terminalapparatus through the communication interface, acquire transformationinformation based on the first photographed image and the first patternimage, control the projection part to project the second pattern imageon the projection surface, and based on receiving a second photographedimage which photographed the projection surface from the externalterminal apparatus through the communication interface, perform colorcalibration based on the characteristic of the projection surface basedon the second photographed image, the second pattern image, and thetransformation information.

The transformation information may include color space transformationinformation, and the processor may be configured to: acquire color spaceinformation corresponding to the first pattern image, and acquire thecolor space transformation information according to the characteristicof the projection part based on the first photographed image and thecolor space information corresponding to the first pattern image.

The processor may be configured to: acquire color space informationcorresponding to the second pattern image, and perform color calibrationaccording to the characteristic of the projection surface based on thesecond photographed image, the color space information corresponding tothe second pattern image, and the color space transformationinformation.

The processor may be configured to: acquire the color spacetransformation information based on RGB information corresponding to thefirst photographed image and XYZ color space information correspondingto the first pattern image.

The transformation information may include a color space transformationmatrix transforming the RGB information into the XYZ color spaceinformation.

The processor may be configured to: transform the RGB informationcorresponding to the second photographed image into the XYZ color spaceinformation based on the transformation information, acquire a colordifference between the XYZ color space information corresponding to thesecond photographed image and the XYZ color space informationcorresponding to the second pattern image, and perform the colorcalibration based on the acquired color difference.

The processor may be configured to change at least one of a gain valueor an offset value related to an RGB signal based on the acquired colordifference.

The processor may, based on identifying that a predetermined objectrelated to the screen member is included in the first photographedimage, be configured to: acquire the transformation information based onthe first photographed image, and based on identifying that apredetermined object related to the screen member is not included in thefirst photographed image, control the projection part to project a userinterface (UI) including information that the screen member is notrecognized.

The first pattern image may include at least one of a white patternimage, a red pattern image, a green pattern image, or a blue patternimage, and the second pattern image may include a white pattern image.

The processor may be configured to control the projection part such thatthe white pattern image among the plurality of pattern images includedin the first pattern image is projected first, and the remaining patternimages are sequentially projected.

A method of controlling an electronic apparatus storing a first patternimage and a second pattern image and communicating with an externalterminal apparatus according to an example embodiment of the disclosureincludes: projecting the first pattern image on a screen member locatedon a projection surface, and based on receiving a first photographedimage which photographed the screen member from the external terminalapparatus, acquiring transformation information based on the firstphotographed image and the first pattern image, projecting the secondpattern image on the projection surface, and based on receiving a secondphotographed image which photographed the projection surface from theexternal terminal apparatus, performing color calibration according tothe characteristic of the projection surface based on the secondphotographed image, the second pattern image, and the transformationinformation.

The transformation information may include color space transformationinformation, and the acquiring the transformation information mayinclude, acquiring color space information corresponding to the firstpattern image, and acquiring the color space transformation informationaccording to the characteristic of a projection part included in theelectronic apparatus based on the first photographed image and the colorspace information corresponding to the first pattern image.

The performing the color calibration may include, acquiring color spaceinformation corresponding to the second pattern image, and performingcolor calibration according to the characteristic of the projectionsurface based on the second photographed image, the color spaceinformation corresponding to the second pattern image, and the colorspace transformation information.

The acquiring the transformation information may include, acquiring thecolor space transformation information based on RGB informationcorresponding to the first photographed image and XYZ color spaceinformation corresponding to the first pattern image.

The transformation information may include a color space transformationmatrix transforming the RGB information into the XYZ color spaceinformation.

The performing the color calibration may include, transforming the RGBinformation corresponding to the second photographed image into the XYZcolor space information based on the transformation information,acquiring a color difference between the XYZ color space informationcorresponding to the second photographed image and the XYZ color spaceinformation corresponding to the second pattern image, and performingthe color calibration based on the acquired color difference.

The performing the color calibration may include, at least one of a gainvalue or an offset value related to an RGB signal may be changed basedon the acquired color difference.

The method may further include: based on identifying that apredetermined object related to the screen member is included in thefirst photographed image, acquiring the transformation information basedon the first photographed image and the first pattern image, and basedon identifying that a predetermined object related to the screen memberis not included in the first photographed image, projecting a userinterface (UI) including information that the screen member is notrecognized.

The first pattern image may include at least one of a white patternimage, a red pattern image, a green pattern image, or a blue patternimage, and the second pattern image may include a white pattern image.

The projecting the first pattern image, includes projecting the whitepattern image among the plurality of pattern images included in thefirst pattern image first, and projecting the remaining pattern imagessequentially.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing detailed description, taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a diagram illustrating an example image projecting operationand an image photographing operation according to various embodiments;

FIG. 2 is a block diagram illustrating an example configuration of anelectronic apparatus according to various embodiments;

FIG. 3 is a block diagram illustrating an example configuration of theelectronic apparatus in FIG. 2 according to various embodiments;

FIG. 4 is a table illustrating various examples of performing colorcalibrating operations according to various embodiments;

FIG. 5 is a diagram illustrating an example color calibrating operationaccording to various embodiments;

FIG. 6 is a diagram illustrating an example color calibrating operationaccording to various embodiments;

FIG. 7 is a diagram illustrating an example color calibrating operationaccording to various embodiments;

FIG. 8 is a diagram illustrating an example color calibrating operationaccording to various embodiments;

FIG. 9 is a diagram illustrating example operations of using a screenmember according to various embodiments;

FIG. 10 is a diagram illustrating example operations of using a screenmember according to various embodiments;

FIG. 11 is a diagram illustrating an example operation of generating acolor space transformation matrix according to various embodiments;

FIG. 12 is a diagram illustrating an example operation of projecting acolor calibration result according to various embodiments;

FIG. 13 is a diagram illustrating an example operation of guiding a userbehavior corresponding to a color calibration result according tovarious embodiments;

FIG. 14 is a diagram illustrating an example operation of comparingprojection before color calibration and projection after colorcalibration according to various embodiments;

FIG. 15 is a diagram illustrating an example operation of comparingprojection before color calibration and projection after colorcalibration according to an various embodiments;

FIG. 16 is a diagram illustrating an example operation of projectinginformation related to a terminal apparatus that can be connected withan electronic apparatus according to various embodiments;

FIG. 17 is a flowchart illustrating an example process of performing anoperation of acquiring a color space transformation matrix and anoperation of performing a color calibrating operation according tovarious embodiments;

FIG. 18 is a flowchart illustrating an example operation of acquiring acolor space transformation matrix according to various embodiments;

FIG. 19 is a flowchart illustrating an example color calibratingoperation according to various embodiments;

FIG. 20 is a flowchart illustrating an example operation of identifyingwhether a predetermined object is included in a screen member accordingto various embodiments;

FIG. 21 is a diagram illustrating an example operation of identifying ascreen member according to various embodiments;

FIG. 22 is a diagram illustrating an example operation of identifying ascreen member according to an various embodiments;

FIG. 23 is a signal flow diagram illustrating an example of performing acolor calibrating operation in an electronic apparatus according tovarious embodiments;

FIG. 24 is a signal flow diagram illustrating an example of performing acolor calibrating operation in a terminal apparatus according to variousembodiments;

FIG. 25 is a flowchart illustrating an example of performing a colorcalibrating operation using streaming data according to variousembodiments;

FIG. 26 is a diagram illustrating an example system including anelectronic apparatus, a terminal apparatus, and a server according tovarious embodiments;

FIG. 27 is a diagram illustrating an example process of acquiring acolor space transformation matrix used in acquiring XYZ color spaceinformation corresponding to a photographed image according to variousembodiments;

FIG. 28 is a diagram illustrating RGB information corresponding to aphotographed image and XYZ color space information corresponding to aphotographed image according to various embodiments; and

FIG. 29 is a flowchart illustrating an example method of controlling anelectronic apparatus according to various embodiments.

DETAILED DESCRIPTION

Hereinafter, the disclosure will be described in greater detail withreference to the accompanying drawings.

As terms used in the embodiments of the disclosure, general terms thatare currently used widely were selected as far as possible, inconsideration of the functions described in the disclosure. However, theterms may vary depending on the intention of those skilled in the artwho work in the pertinent technical field or previous court decisions,emergence of new technologies, etc. Also, in particular cases, there maybe arbitrarily selected terms, and in such cases, the meaning of theterms will be described in detail in the relevant descriptions in thedisclosure. Accordingly, the terms used in the disclosure should bedefined based on the meaning of the terms and the overall content of thedisclosure, but not just based on the names of the terms.

In the disclosure, expressions such as “have,” “may have,” “include,”and “may include” denote the existence of such characteristics (e.g.:elements such as numbers, functions, operations, and components), and donot exclude the existence of additional characteristics.

In addition, the expression “at least one of A and/or B” should beinterpreted to refer to any one of “A” or “B” or “A and B.”

The expressions “first,” “second” and the like used in this disclosuremay be used to describe various elements regardless of any order and/ordegree of importance. In addition, such expressions are used only todistinguish one element from another element, and are not intended tolimit the elements.

The description in the disclosure that one element (e.g., a firstelement) is “(operatively or communicatively) coupled with/to” or“connected to” another element (e.g., a second element) should beinterpreted to include both the case where the one element is directlycoupled to the another element, and the case where the one element iscoupled to the another element through still another element (e.g., athird element).

In addition, singular expressions include plural expressions, as long asthey do not conflict in context. In the disclosure, terms such as“include” and “consist of” should be construed as designating that thereare such characteristics, numbers, steps, operations, elements,components, or a combination thereof described in the disclosure, butnot as excluding in advance the existence or possibility of adding oneor more of other characteristics, numbers, steps, operations, elements,components, or a combination thereof.

In the disclosure, “a module” or “a part” may perform at least onefunction or operation, and may be implemented as hardware or software,or as a combination of hardware and software. A plurality of “modules”or “parts” may be integrated into at least one module and implemented asat least one processor (not shown), except “modules” or “parts” whichneed to be implemented as specific hardware.

In the disclosure, the term “user” may refer to a person who uses anelectronic apparatus or an apparatus using an electronic apparatus(e.g.: an artificial intelligence electronic apparatus).

Hereinafter, various example embodiments of the disclosure will bedescribed in greater detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating an example image projecting operationand an image photographing operation according to various embodiments.

The electronic apparatus 100 may refer, for example, to variousapparatuses performing a function of a projector. The electronicapparatus 100 may include a projection part (e.g., projector) 120. Theprojection part 120 may refer, for example, to hardware projecting aspecific image. For example, the projection part 120 may refer, forexample, to an image projection lens which may be referred tohereinafter as, or may be used interchangeably with, the term projector.

The electronic apparatus 100 may project an image 20 on a screen 10using the projection part 120.

A terminal apparatus 200 may photograph the projected image 20. Theterminal apparatus 200 may include a camera 210. The terminal apparatus200 may photograph the image 20 using the camera 210.

According to an embodiment, the terminal apparatus 200 may photograph aprojection surface including the projected image 20. A user mayphotograph not only the area wherein the projected image 20 isprojected, but also areas including the other areas. Accordingly, theelectronic apparatus 100 may selectively use only the area wherein theprojected image 20 is included in the photographed image photographed bythe user.

According to an embodiment, the terminal apparatus 200 may photographonly the projected image 20. As information that is actually needed forcolor calibration is the part corresponding to the projected image 20,the user may photograph only the projected image 20. The electronicapparatus 100 may provide information for guiding the user to photographonly the projected image 20.

FIG. 2 is a block diagram illustrating an example configuration of theelectronic apparatus 100 according to various embodiments.

Referring to FIG. 2, the electronic apparatus 100 may include at leastone of a memory 110, a projection part (e.g., projector or projectionlens) 120, a processor (e.g., including processing circuitry) 130 and/ora communication interface (e.g. including communication circuitry) 150.

In the memory 110, at least one instruction related to the electronicapparatus 100 may be stored. In the memory 110, an operating system(O/S) for driving the electronic apparatus 100 may be stored. Inaddition, in the memory 110, various kinds of software programs orapplications for the electronic apparatus 100 to operate according tothe various embodiments of the disclosure may be stored. The memory 110may include a semiconductor memory such as a flash memory or a magneticstorage medium such as a hard disk, etc.

In the memory 110, various kinds of software modules for the electronicapparatus 100 to operate according to the various embodiments of thedisclosure may be stored, and the processor 130 may control theoperations of the electronic apparatus 100 by executing the variouskinds of software modules stored in the memory 110. For example, thememory 110 may be accessed by the processor 130, andreading/recording/correcting/deleting/updating, etc. of data by theprocessor 130 may be performed.

The memory 110 may include a ROM (not shown) and a RAM (not shown)inside the processor 130, or a memory card (not shown) (e.g., a micro SDcard, a memory stick) installed on the electronic apparatus 100.

The memory 110 may store information on a first pattern image and asecond pattern image. For example, the memory 110 may store the firstpattern image, RGB information corresponding to the first pattern image,color space information corresponding to the first pattern image (e.g.,XYZ color space information), and color space information correspondingto the second pattern image (e.g., XYZ color space information).

The projection part 120 may include a projector or projection lens andoutput an image to be output from the electronic apparatus 100 to theprojection surface. The projection part 120 may include a projectionlens.

The projection part 120 may perform a function of outputting an image tothe screen (or the projection surface). The projection part 120 is acomponent projecting an image to the outside. The projection part 120according to an embodiment of the disclosure may be implemented invarious projection methods (e.g., a cathode-ray tube (CRT) method, aliquid crystal display (LCD) method, a digital light processing (DLP)method, a laser method, etc.).

The projection part 120 may perform various functions for adjusting anoutput image by control by the processor 130. For example, theprojection part 120 may perform functions such as zoom, keystone, quickcorner (four corner) keystone, lens shift, etc.

The processor 130 may include various processing circuitry and performoverall controlling operations of the electronic apparatus 100. Forexample, the processor 130 performs a function of controlling theoverall operations of the electronic apparatus 100.

The processor 130 may be implemented as a digital signal processor (DSP)processing digital signals, a microprocessor, and a time controller(TCON). However, the disclosure is not limited thereto, and theprocessor 130 may include one or more of a central processing unit(CPU), a micro controller unit (MCU), a micro processing unit (MPU), acontroller, an application processor (AP), a graphics-processing unit(GPU) or a communication processor (CP), and an ARM processor, or may bedefined by the terms. Also, the processor 130 may be implemented as asystem on chip (SoC) having a processing algorithm stored therein orlarge scale integration (LSI), or in the form of a field programmablegate array (FPGA). The processor 130 may perform various functions byexecuting computer executable instructions stored in the memory 110.

The processor 130 may control the projection part 120 to project thefirst pattern image on a screen member 30 (refer to FIG. 5) located onthe projection surface, and if a first photographed image whichphotographed the screen member 30 is received from an external terminalapparatus through a communication interface 150, the processor 130 mayacquire transformation information based on the first photographed imageand the first pattern image, control the projection part 120 to projectthe second pattern image on the projection surface, and if a secondphotographed image which photographed the projection surface is receivedfrom the external terminal apparatus through the communication interface150, the processor 130 may perform color calibration according to thecharacteristic of the projection surface based on the secondphotographed image, the second pattern image, and the transformationinformation.

The transformation information may include color space transformationinformation, and the processor 130 may acquire color space informationcorresponding to the first pattern image, and acquire the color spacetransformation information according to the characteristic of theprojection part 120 based on the first photographed image and the colorspace information corresponding to the first pattern image. Here, thememory 110 may store the color space information corresponding to thefirst pattern image.

The processor 130 may acquire color space information corresponding tothe second pattern image, and perform color calibration according to thecharacteristic of the projection surface based on the secondphotographed image, the color space information corresponding to thesecond pattern image, and the color space transformation information.The memory 110 may store the color space information corresponding tothe second pattern image.

The processor 130 may control the projection part 120 to project thefirst pattern image on the screen member 30 (refer to FIG. 5). Theremay, for example, be two methods of using the screen member 30. As anexample, the screen member 30 may be a member directly installed by auser. Explanation in this regard will be described in greater detailbelow with reference to FIG. 5 and FIG. 7. As another example, thescreen member 30 may be a member included in (or attached on) theelectronic apparatus 100. Explanation in this regard will be describedin greater detail below with reference to FIG. 6 and FIG. 8.

The screen member 30 may refer, for example, to a white reflector, andit may refer, for example, to a reflector having the standard whitecolor provided by the manufacturer of the electronic apparatus 100. Whena reflector manufactured officially by the manufacturer of theelectronic apparatus 100 is used, correct color space transformationinformation can be acquired. An operation of identifying whether anofficial (or a genuine) reflector is installed will be described ingreater detail below with reference to FIG. 20 to FIG. 22. The screenmember 30 may be described as an official reflector, a genuinereflector, a standard reflector, etc.

The screen member 30 may, for example, be a member satisfying at leastone standard among the standard color, the standard specification, andthe standard material. The screen member 30 may be a plane.

The first pattern image may be a test pattern image projected by theprojection part 120. The first pattern image may include a white patternimage, a red pattern image, a green pattern image, and a blue patternimage, or the like.

For example, after the first pattern image is projected on the screenmember 30, the processor 130 may acquire the first photographed imagewhich photographed the screen member 30 on which the first pattern imageis projected. The first photographed image may include the screen member30 on which the first pattern image is projected. There may be variousmethods of acquiring a photographed image. As an example, a photographedimage may be photographed by the camera of the terminal apparatus 200,and the electronic apparatus 100 may receive the photographed image fromthe terminal apparatus 200. Explanation in this regard will be describedin greater detail below with reference to FIG. 5 and FIG. 6. As anotherexample, a photographed image may be photographed by the camera attachedon the electronic apparatus 100, and the processor 130 may acquire thephotographed image by the camera of the electronic apparatus 100.Explanation in this regard will be described in greater detail belowwith reference to FIG. 7 and FIG. 8.

The characteristic of the projection part 120 may refer, for example, toa hardware attribute related to the projection part 120. For example,the characteristic of the projection part 120 may include informationrelated to the performance of the projection lens included in theprojection part 120. Accordingly, the color space transformationinformation may vary according to the hardware attribute of theprojection part 120. The processor 130 may acquire color spacetransformation information appropriate for the projection part 120 basedon the color space information corresponding to the first photographedimage and the first pattern image.

The processor 130 may perform a pre-calibration operation using thewhite pattern image in the first pattern image. For example, theprocessor 130 may project the white pattern image on the screen member30, and acquire a photographed image including the screen member 30 onwhich the white pattern image is projected (a photographed imageincluding the white pattern image). The processor 130 may compare thephotographed image including the white pattern image and the whitepattern image (the original image), and adjust the sensitivity of theprojection part 120. For example, the processor 130 may adjust a setvalue related to the performance of the projection lens included in theprojection part 120. For example, the processor 130 may change a setvalue related to the shutter speed or the aperture.

The processor 130 may perform color calibration using the red patternimage, the green pattern image, and the blue pattern image in the firstpattern image.

The processor 130 may project the red pattern image on the screen member30, and acquire a photographed image including the screen member 30 onwhich the red pattern image is projected (a photographed image includingthe red pattern image). The processor 130 may compare the photographedimage including the red pattern image and the red pattern image (theoriginal image), and acquire at least one simultaneous equation relatedto the red pattern image.

The processor 130 may project the green pattern image on the screenmember 30, and acquire a photographed image including the screen member30 on which the green pattern image is projected (a photographed imageincluding the green pattern image). The processor 130 may compare thephotographed image including the green pattern image and the greenpattern image (the original image), and acquire at least onesimultaneous equation related to the green pattern image.

The processor 130 may project the blue pattern image on the screenmember 30, and acquire a photographed image including the screen member30 on which the blue pattern image is projected (a photographed imageincluding the blue pattern image). The processor 130 may compare thephotographed image including the blue pattern image and the blue patternimage (the original image), and acquire at least one simultaneousequation related to the blue pattern image.

The processor 130 may acquire color space transformation information (acolor space transformation matrix) based on the at least onesimultaneous equation related to the red pattern image, the at least onesimultaneous equation related to the green pattern image, and the atleast one simultaneous equation related to the blue pattern image.

Explanation related to the simultaneous equations will be described ingreater detail below with reference to FIG. 27.

The processor 130 may acquire the color space transformation informationbased on information related to the first photographed image includingthe first pattern image projected on the screen member 30 and the firstpattern image stored in the memory 110 in advance. The informationrelated to the first pattern image may include the color spaceinformation corresponding to the first pattern image.

The color space transformation information may refer, for example, to amatrix for transforming general data into color space data.

The processor 130 may acquire the color space transformation informationbased on RGB information corresponding to the first photographed imageand XYZ color space information corresponding to the first patternimage.

There may be various definitions defining the color space. Also, whileXYZ was illustrated in the aforementioned description, other definitionsof the color space may be used depending on implementation examples.

The color space transformation information may be a color spacetransformation matrix transforming the RGB information into the XYZcolor space information. A specific operation of acquiring the colorspace transformation matrix will be described in greater detail belowwith reference to FIG. 17 and FIG. 18. A specific calculating operationof acquiring the color space transformation matrix will be described ingreater detail below with reference to FIG. 27 and FIG. 28.

After the color space transformation matrix is acquired, the processor130 may project the second pattern image on the screen 10. For example,after the color space transformation matrix is acquired, the processor130 may project the second pattern image on the screen 10 wherein thescreen member 30 does not exist. The processor 130 may acquire the firstphotographed image which photographed the screen member 30 and acquirethe color space transformation matrix, and acquire the secondphotographed image which photographed the screen 10 wherein the screenmember 30 is not arranged, and perform color calibration according tothe projection surface. For example, for resolving the problem that thecolor of the original image is not expressed as it is due to the colorof the screen 10, the processor 130 may photograph the screen 10 andperform color calibration. As a result, the processor 130 may output aprojection image appropriate for the projection surface through colorcalibration. When the projection surface varies, color calibration mayalso vary.

As an example, the first pattern image and the second pattern image maybe the same. The first pattern image and the second pattern image may bethe same white pattern images. As another example, the first patternimage and the second pattern image may be different. The first patternimage may be one of the red pattern image, the green pattern image, orthe blue pattern image, and the second pattern image may be the whitepattern image.

The processor 130 may acquire the second photographed image whichphotographed the screen 10 in a state wherein the screen member 30 isnot arranged. For resolving the problem that the color is distorted dueto the color of the screen 10, the processor 130 may perform colorcalibration based on the second photographed image.

The processor 130 may acquire RGB information corresponding to thesecond photographed image.

The processor 130 may transform the RGB information corresponding to thesecond photographed image into the XYZ color space information based onthe color space transformation information (e.g., the color spacetransformation matrix), acquire a color difference between the XYZ colorspace information corresponding to the second photographed image and theXYZ color space information corresponding to the second pattern image,and perform the color calibration based on the acquired colordifference.

The processor 130 may change at least one of a gain value or an offsetvalue related to an RGB signal based on the acquired color difference.

The processor 130 may change the gain value related to the RGB signal byperforming a color calibrating operation. The gain value may refer, forexample, to an element for adjusting the output value of the RGB signalthrough a multiplication (or a division) operation.

The processor 130 may change the offset related to the RGB signal byperforming a color calibrating operation. The offset may refer, forexample, to an element for adjusting the output value of the RGB signalthrough an addition (or a subtraction) operation.

If the output value of the RGB signal is adjusted, at least one of theluminance, the contrast, or the color of the projected image may bechanged, and an RGB signal appropriate for the screen (or the projectionsurface) may be output (or projected).

The electronic apparatus 100 according to an embodiment of thedisclosure may output an RGB signal.

The electronic apparatus 100 according to an embodiment of thedisclosure may output signals in different forms other than an RGBsignal. For example, the electronic apparatus 100 may output an RGBWsignal and an RGBY signal. The electronic apparatus 100 may additionallyoutput a signal of at least one of yellow, cyan, or magenta other thanan RGB signal. Accordingly, the electronic apparatus 100 does notnecessarily control only RGB by color calibration, but it may calibrateadditional pixels (or sub-pixels).

The processor 130 may perform an operation of identifying whether thescreen member 30 is a standard screen member 30 made by themanufacturer. This is because, if the screen member 30 is not a standardscreen member 30, some errors may occur in the color spacetransformation matrix.

If it is identified that a predetermined (e.g., specified) objectrelated to the screen member 30 is included in the first photographedimage, the processor 130 may acquire the color space transformationinformation based on the first photographed image, and if it isidentified that a predetermined object related to the screen member 30is not included in the first photographed image, the processor 130 maycontrol the projection part 120 to project a UI including informationthat the screen member 30 is not recognized. Specific operations in thisregard will be described in greater detail below with reference to FIG.20 to FIG. 22.

Example methods for the electronic apparatus 100 to acquire aphotographed image may be divided into two.

According to an embodiment, the electronic apparatus 100 may receive aphotographed image from the terminal apparatus 200 which may be anexternal apparatus. The electronic apparatus 100 may further include acommunication interface 150 including various communication circuitry,and the processor 130 may control the communication interface to receivethe first photographed image and the second photographed image from theexternal terminal apparatus 200.

According to an embodiment, the electronic apparatus 100 may acquire aphotographed image using a camera 140 installed inside the electronicapparatus 100. The electronic apparatus 100 may further include a camera140, and the processor 130 may control the camera 140 to acquire thefirst photographed image and the second photographed image.

The first pattern image may include at least one of a white patternimage, a red pattern image, a green pattern image, or a blue patternimage, and the second pattern image may include a white pattern image.

The processor 130 may control the projection part 120 such that thewhite pattern image among the plurality of pattern images included inthe first pattern image is projected first, and the remaining patternimages are sequentially projected.

The first pattern image may be an image projected on the screen member30, and the processor 130 may preferentially project the white patternimage. After acquiring a photographed image which photographed thescreen member 30 on which the white pattern image is projected, theprocessor 130 may project the red pattern image, the green patternimage, or the blue pattern image in any order. If the white patternimage is output first, the projection order of the red pattern image,the green pattern image, or the blue pattern image may vary according tothe user setting.

A reason that the second pattern image is output only as the whitepattern image, unlike the first pattern image, is because it isadvantageous for color calibration to use the white color forconsidering the color of the screen 10.

In the above, simple components of the electronic apparatus 100 wereillustrated and described, but in actual implementation, variouscomponents may additionally be included. Explanation in this regard willbe made in greater detail below with reference to FIG. 3.

There may be various methods of outputting the first pattern image andthe second pattern image.

According to an embodiment, the first pattern image and the secondpattern image may have been stored in the memory 110 of the firstpattern image. In case a predetermined (e.g., specified) event occurs,the electronic apparatus 100 may output the first pattern image storedin the memory 110 or output the second pattern image stored in thememory 110.

According to an embodiment, the first pattern image and the secondpattern image may be provided from the terminal apparatus 200. Forexample, if a predetermined (e.g., specified) control signal is receivedfrom the electronic apparatus 100, the terminal apparatus 200 maytransmit the first pattern image or the second pattern image to theelectronic apparatus 100 in real time. The predetermined control signalmay be a signal requesting the first pattern image or a signalrequesting the second pattern image. The electronic apparatus 100 mayoutput the first pattern image or the second pattern image received fromthe terminal apparatus 200.

According to an embodiment, the first pattern image and the secondpattern image may be provided from the server 300. For example, if apredetermined control signal is received from the electronic apparatus100, the server 300 may transmit the first pattern image or the secondpattern image to the electronic apparatus 100. The predetermined controlsignal may be a signal requesting the first pattern image or a signalrequesting the second pattern image. The electronic apparatus 100 mayoutput the first pattern image or the second pattern image received fromthe terminal apparatus 200.

There may be various methods of acquiring color space informationcorresponding to the first pattern image.

According to an embodiment, color space information corresponding to thefirst pattern image may have been stored in the memory 110. In case apredetermined event occurred, the electronic apparatus 100 may acquirethe color space information corresponding to the first pattern imagestored in the memory 110.

According to an embodiment, the color space information corresponding tothe first pattern image may be provided from the server 300. Forexample, if a predetermined control signal is received, the server 300may transmit the color space information corresponding to the firstpattern image to the electronic apparatus 100. The predetermined controlsignal may be a signal requesting the color space informationcorresponding to the first pattern image. The electronic apparatus 100may receive the color space information corresponding to the firstpattern image.

There may be various time points of outputting the first pattern imageand the second pattern image.

According to an embodiment, the electronic apparatus 100 may output thefirst pattern image first, and output the second pattern image after thefirst pattern image is output. For example, the electronic apparatus 100may output the first pattern image in a state wherein the screen member30 is installed, and output the second pattern image in a state whereinthe screen member 30 is not installed.

According to an embodiment, the electronic apparatus 100 may output thefirst pattern image and the second pattern image simultaneously. Forexample, the electronic apparatus 100 may output the first pattern imageand the second pattern image simultaneously in a state wherein thescreen member 30 is installed. The electronic apparatus 100 may outputthe first pattern image in the first area wherein the screen member 30is located among the entire areas of the projection surface, and outputthe second pattern image in the second area wherein the screen member 30is not located among the entire areas of the projection surface. Theelectronic apparatus 100 may acquire a photographed image whichphotographed the projection surface. The photographed image may includethe first pattern image output in the first area and the second patternimage output in the second area. The electronic apparatus 100 mayacquire transformation information (e.g., a color space transformationmatrix) based on the first pattern image output in the first area, andperform color calibration based on the second pattern image output inthe second area. The electronic apparatus 100 may acquire thetransformation information (e.g., a color space transformation matrix)based on the original image of the first pattern image and the firstpattern image included in the photographed image, and at the same time,perform color calibration according to the characteristic of theprojection surface based on the original image of the second patternimage, the second pattern image included in the photographed image, andthe transformation information (e.g., a color space transformationmatrix).

The electronic apparatus 100 may be implemented in a form wherein thefirst pattern image and the second pattern image are the same, and colorcalibration is performed with one photographed image. For example, theelectronic apparatus 100 may output the pattern image in a state whereinthe screen member 30 is included. The electronic apparatus 100 mayoutput the pattern image in both of the first area wherein the screenmember 30 is included and the second area wherein the screen member 30is not included. For example, some of the pattern image may be output inthe first area (including the screen member 30), and some of the patternimage may be output in the second area (not including the screen member30). The electronic apparatus 100 may acquire the transformationinformation (e.g., a color space transformation matrix) based on theoriginal image of the pattern image and the pattern image included inthe photographed image, and at the same time, perform color calibrationaccording to the characteristic of the projection surface based on theoriginal image of the pattern image, the pattern image included in thephotographed image, and the transformation information (e.g., a colorspace transformation matrix).

FIG. 3 is a block diagram illustrating an example configuration of theelectronic apparatus 100 in FIG. 2 according to various embodiments.

Referring to FIG. 3, the electronic apparatus 100 may include at leastone of a memory 110, a projection part (e.g., including a projector orprojection lens) 120, a processor (e.g., including processing circuitry)130, a camera 140, a communication interface (e.g., includingcommunication circuitry) 150, a manipulation interface (e.g., includingvarious circuitry) 161, an input/output interface (e.g., includingvarious input/output circuitry) 162, a speaker 170, a microphone 180, ora power part (e.g., including power management circuitry) 190.

Among the operations of the memory 110, the projection part 120, and theprocessor 130, regarding operations identical or similar to what weredescribed above, overlapping explanation may not be repeated here.

The camera 140 is a component for photographing a subject and generatinga photographed image, and here, the photographed image is a conceptincluding both of a moving image and a still image. The camera 140 mayacquire an image for at least one external apparatus, and it may beimplemented as a camera, a lens, an infrared sensor, etc.

The camera 140 may include a lens and an image sensor. As types of thelens, there are a general generic-purpose lens, a wide-angle lens, azoom lens, etc., and the type may be determined according to the type,the characteristic, the use environment, etc. of the electronicapparatus 100. As the image sensor, a complementary metal oxidesemiconductor (CMOS) and a charge coupled device (CCD), etc. may beused.

The camera 140 outputs an incident light as an image signal. Forexample, the camera 140 may include a lens, pixels, and an AD converter.The lens gathers the light of a subject and makes an optical imageformed in a photographing area, and the pixels may output the lightintroduced through the lens as an image signal in an analog form. The ADconverter may convert the image signal in an analog form into an imagesignal in a digital form, and output the signal. For example, the camera140 is arranged to photograph the front surface direction of theelectronic apparatus 100, and photograph a user who exists on the frontsurface of the electronic apparatus 100, and generate a photographedimage.

The communication interface 150 is a component performing communicationwith external apparatuses in various types according to communicationmethods in various types. The communication interface 150 may includevarious communication circuitry including, for example, a wirelesscommunication module or a wired communication module. Each communicationmodule may be implemented in a form of at least one hardware chip.

A wireless communication module may be a module communicating with anexternal apparatus wirelessly. For example, the wireless communicationmodule may include at least one module among a Wi-Fi module, a Bluetoothmodule, an infrared communication module, or other communicationmodules.

A Wi-Fi module and a Bluetooth module may perform communication using aWi-Fi method and a Bluetooth method, respectively. In the case of usinga Wi-Fi module or a Bluetooth module, various types of connectioninformation such as an SSID and a session key is transmitted andreceived first, and connection of communication is performed using theinformation, and various types of information can be transmitted andreceived thereafter.

An infrared communication module performs communication according to aninfrared Data Association (IrDA) technology of transmitting data to anear field wirelessly using infrared rays between visible rays andmillimeter waves.

Other communication modules may include at least one communication chipthat performs communication according to various wireless communicationprotocols such as Zigbee, 3rd Generation (3G), 3rd GenerationPartnership Project (3GPP), Long Term Evolution (LTE), LTE Advanced(LTE-A), 4th Generation (4G), 5th Generation (5G), etc. other than theaforementioned communication methods.

A wired communication module may be a module communicating with anexternal apparatus via wire. For example, a wired communication modulemay include at least one of a local area network (LAN) module, anEthernet module, a pair cable, a coaxial cable, an optical fiber cable,or an ultra wide-band (UWB) module.

The manipulation interface 161 may include various circuitry and beimplemented as an apparatus like a button, a touch pad, a mouse, and akeyboard, or as a touch screen that can perform both of theaforementioned display function and a manipulation input functiontogether. A button may be various types of buttons such as a mechanicalbutton, a touch pad, a wheel, etc. formed in any areas such as the frontsurface part, the side surface part, the rear surface part, etc. of theexterior of the main body of the electronic apparatus 100.

The input/output interface 162 may be an interface including variousinput/output circuitry including of any one of a high definitionmultimedia interface (HDMI), a mobile high-definition link (MHL), auniversal serial bus (USB), a display port (DP), a Thunderbolt, a videographics array (VGA) port, an RGB port, a D-subminiature (D-SUB), or adigital visual interface (DVI). The input/output interface 162 may inputor output at least one of an audio signal or a video signal. Dependingon implementation examples, the input/output interface 162 may include aport outputting only audio signals and a port outputting only videosignals as separate ports, or it may be implemented as one port thatinputs and outputs both audio signals and video signals. The electronicapparatus 100 may transmit at least one of an audio signal or a videosignal to an external apparatus (e.g., an external display apparatus oran external speaker) through the input/output interface 162.Specifically, an output port included in the input/output interface 162may be connected with an external apparatus, and the electronicapparatus 100 may transmit at least one of an audio signal or a videosignal to the external apparatus through the output port.

The input/output interface 162 may be connected with the communicationinterface. The input/output interface 162 may transmit informationreceived from an external apparatus to the communication interface, ortransmit information received through the communication interface to anexternal apparatus.

The speaker 170 may be a component that outputs not only various kindsof audio data but also various kinds of notification sounds or voicemessage, etc.

The electronic apparatus 100 may include a microphone 180.

The microphone 180 is a component for receiving input of a user voice orother sounds and converting them into audio data. The microphone 180 mayreceive a voice of a user in an activated state. For example, themicrophone 180 may be formed as an integrated type on the upper side orthe front surface direction, the side surface direction, etc. of theelectronic apparatus 100. The microphone 180 may include variouscomponents such as a microphone collecting a user voice in an analogform, an amp circuit amplifying the collected user voice, an A/Dconversion circuit that samples the amplified user voice and convertsthe user voice into a digital signal, a filter circuit that removesnoise components from the converted digital signal, etc.

The power part 190 may include various power management circuitry and beprovided with power from the outside, and provide power to the variouscomponents of the electronic apparatus 100. The power part 190 accordingto an embodiment of the disclosure may be provided with power throughvarious methods. The power part 190 may be provided with power using aDC power code of 220V. However, the disclosure is not limited thereto,and the electronic apparatus 100 may be provided with power using a USBpower code or provided with power using a wireless charging method.

The electronic apparatus 100 may further include a display (not shown).

The display (not shown) may be implemented as displays in various formssuch as a liquid crystal display (LCD), an organic light emitting diodes(OLED) display, a plasma display panel (PDP), etc. Inside the display(not shown), driving circuits that may be implemented in forms such asan a-si TFT, a low temperature poly silicon (LTPS) TFT, an organic TFT(OTFT), etc., a backlight unit, etc. may also be included. The display(not shown) may be implemented as a touch screen combined with a touchsensor, a flexible display, a 3D display, etc. The display (not shown)according to an embodiment of the disclosure may include not only adisplay panel outputting images, but also a bezel housing the displaypanel. In particular, a bezel according to an embodiment of thedisclosure may include a touch sensor (not shown) for detecting userinteractions.

FIG. 4 is a table illustrating various examples of performing colorcalibrating operations according to various embodiments.

Referring to the table 410 in FIG. 4, in performing a color calibratingoperation, various embodiments may exist according to a subjectacquiring an image and a method of using the screen member. The subjectacquiring an image may be the terminal apparatus 200 or the electronicapparatus 100. The method of using the screen member may be a method ofinstallation by a user or a method of accommodation inside theelectronic apparatus.

The method of installation by a user may refer, for example, to a userinstalling a separate screen member 30 on the projection surfacedirectly. The method of accommodation inside the electronic apparatusmay refer, for example, to a method wherein the screen member 30 isaccommodated inside the electronic apparatus 100, and the screen member30 is used automatically.

According to the first embodiment, a color calibrating operation may beperformed based on an operation of the terminal apparatus 200 ofacquiring an image and an operation of a user of directly installing thescreen member 30. Specific explanation in this regard will be describedin greater detail below with reference to FIG. 5.

According to the second embodiment, a color calibrating operation may beperformed based on an operation of the terminal apparatus 200 ofacquiring an image and an operation of using the screen member 30accommodated inside the electronic apparatus 100. Specific explanationin this regard will be described in FIG. 6.

According to the third embodiment, a color calibrating operation may beperformed based on an operation of the electronic apparatus 100 ofacquiring an image and an operation of a user of directly installing thescreen member 30. Specific explanation in this regard will be describedin FIG. 7.

According to the fourth embodiment, a color calibrating operation may beperformed based on an operation of the electronic apparatus 100 ofacquiring an image and an operation of using the screen member 30accommodated inside the electronic apparatus 100. Specific explanationin this regard will be described in FIG. 8. While the terms “firstembodiment”, “second embodiment”, “third embodiment”, “fourthembodiment”, or the like are used herein for ease of description, itwill be understood that the various embodiments are not necessarilymutually exclusive and that any of the embodiment(s) described hereinmay be used in conjunction with any other embodiment(s) describedherein.

FIG. 5 is a diagram illustrating an example color calibrating operationaccording to various embodiments.

Referring to FIG. 5, a user may install the screen member (e.g., areflector) 30 on the screen 10. After the screen member 30 is installedon the screen 10, the electronic apparatus 100 may project the firstpattern image 510 on the screen member 30. The terminal apparatus 200may photograph the first pattern image 510 projected on the screenmember 30.

FIG. 6 is a diagram illustrating an example color calibrating operationaccording to various embodiments.

Referring to FIG. 6, the electronic apparatus 100 may include the screenmember 30. As an example, the screen member 30 included in theelectronic apparatus 100 may be spread as in FIG. 6 by a user'smanipulation. As another example, the screen member 30 included in theelectronic apparatus 100 may be spread as in FIG. 6 by a motor (notshown). In a state wherein the screen member 30 is spread, theelectronic apparatus 100 may project the first pattern image 610 on thescreen member 30. The terminal apparatus 200 may photograph the firstpattern image 610 projected on the screen member 30.

FIG. 7 is a diagram illustrating an example color calibrating operationaccording to various embodiments.

Referring to FIG. 7, a user may install the screen member 30 on thescreen 10. After the screen member 30 is installed on the screen 10, theelectronic apparatus 100 may project the first pattern image 710 on thescreen member 30 using the projection part 120. The electronic apparatus100 may photograph the first pattern image 710 projected on the screenmember 30 using the camera 140 included in the electronic apparatus 100.

FIG. 8 is a diagram illustrating an example color calibrating operationaccording to various embodiments.

Referring to FIG. 8, the electronic apparatus 100 may include the screenmember 30. As an example, the screen member 30 included in theelectronic apparatus 100 may be spread as in FIG. 8 by a user'smanipulation. As another example, the screen member 30 included in theelectronic apparatus 100 may be spread as in FIG. 8 by a motor (notshown). In a state wherein the screen member 30 is spread, theelectronic apparatus 100 may project the first pattern image 810 on thescreen member 30. Then, the electronic apparatus 100 may photograph thefirst pattern image 810 projected on the screen member 30.

FIG. 9 is a diagram illustrating example operations of using the screenmember 30 according to various embodiments.

Referring to FIG. 9, the electronic apparatus 100 may project an imagefor guiding the arrangement location of the screen member through theprojection part 120. For example, the guide image projected by theelectronic apparatus 100 may include at least one of a text 910 or a UI920 for guiding the arrangement of the screen member 30. A user mayeasily figure out the location of the screen member 30 based on theprojected UI 920, and locate the screen member 30 as it is on theprojected UI 920.

FIG. 10 is a diagram illustrating example operations of using the screenmember 30 according to various embodiments.

Referring to FIG. 10, the electronic apparatus 100 may include thescreen member 30. As an example, the screen member 30 included in theelectronic apparatus 100 may be spread by a user's manipulation. Asanother example, the screen member 30 included in the electronicapparatus 100 may be spread by a motor (not shown).

Referring to FIG. 10, 1010 expresses a state wherein the screen member30 is folded. 1020 expresses a state wherein the screen member 30 isspread. 1030 expresses a state wherein the projection part 120 or thecamera 140 is arranged on the electronic apparatus 100.

The screen member 30 disclosed in FIG. 10 is illustrated to be attachedon the electronic apparatus 100. The screen member 30 may be attached toor detached from the electronic apparatus 100, and a user may separatethe screen member 30 from the electronic apparatus 100 and attach it onthe projection surface. Accordingly, a user does not have to keep thescreen member 30 separately, and a user may keep the screen member 30easily by fixing it on the electronic apparatus 100.

FIG. 11 is a diagram illustrating an example operation of generating acolor space transformation matrix according to various embodiments.

Referring to FIG. 11, the electronic apparatus 100 may perform anoperation of acquiring a color space transformation matrix.

Referring to the FIG. 11, 1110, the electronic apparatus 100 may storecolor space information corresponding to the first pattern image 1111.The color space information may refer, for example, to a CIE XYZ colorspace. CIE may refer, for example, to the International Commission onIllumination (Commission Internationale de l'Eclairage). For example,the electronic apparatus 100 may project the first pattern image 1111 onthe screen member 30. Then, a professional measurement apparatus 1100may photograph the first pattern image 1111 projected on the screenmember 30. Based on the image which photographed the first pattern image1111, the electronic apparatus 100 may acquire color space informationcorresponding to the first pattern image 1111. The acquired color spaceinformation may be standard information. As the professional measurementapparatus 1100 is an apparatus having high precision, it would bedifficult for general consumers to own it. Accordingly, there may be adifference between an image photographed by the professional measurementapparatus 1100 and an image photographed by a general terminal apparatus200.

Referring to 1120, the electronic apparatus 100 may acquire RGBinformation corresponding to the first pattern image 1121. For example,the electronic apparatus 100 may project the first pattern image 1121 onthe screen member 30, and the terminal apparatus 200 may photograph theprojected first pattern image 1121. Here, the first pattern image 1121may be the same as the first pattern image 1111 in the embodiment 1110.The electronic apparatus 100 may acquire RGB information correspondingto the first pattern image photographed by the terminal apparatus 200.

The electronic apparatus 100 may acquire a color space transformationmatrix based on the CIE XYZ color space information corresponding to thefirst pattern image 1111 acquired through the embodiment 1110 and theRGB information corresponding to the first photographed image acquiredthrough the embodiment 1120.

FIG. 12 is a diagram illustrating an example operation of projecting acolor calibration result according to various embodiments.

Referring to FIG. 12, after performing a color calibrating operation,the electronic apparatus 100 may project a result image 1210 on thescreen 10. The result image may include information indicating to whichratio the color calibration is possible for the projection surfacecompared to the target level. For example, if the target level is 100%,the electronic apparatus 100 may project the result image 1210 includinginformation that the color calibration is possible to 90% of the targetlevel on the screen 10.

FIG. 13 is a diagram illustrating an example operation of guiding a userbehavior corresponding to a color calibration result according tovarious embodiments.

Referring to FIG. 13, after performing a color calibrating operation,the electronic apparatus 100 may project a result image 1310 includingthe color calibration result 1311 and information for guiding a userbehavior 1312 on the screen 10. The color calibration result 1311 mayrefer, for example, to information indicating to which ratio the colorcalibration is possible compared to the target level. The informationfor guiding a user behavior 1312 may include an action that a user cantake for making the color calibration result better. For example, theinformation for guiding a user behavior 1312 may include informationguiding to make the lighting dark for enhancing the precision of thecolor calibration. In case the lighting is made to be dark, the firstpattern image or the second pattern image may be recognized moreclearly. Accordingly, the electronic apparatus 100 may provideinformation guiding to make the lighting dark for enhancing theprecision of the color calibration to a user.

FIG. 14 is a diagram illustrating an example operation of comparingprojection before color calibration and projection after colorcalibration according to various embodiments.

Referring to FIG. 14, 1401 may indicate the electronic apparatus 100projecting a result image 1410 according to the setting before colorcalibration. The electronic apparatus 100 may project the result image1410 after the color calibrating operation. The result image 1410 mayinclude at least one of a UI 1411 indicating the color calibrationresult, a UI 1412 for guiding to project the result image according tothe setting before the color calibration, a UI 1413 for guiding toproject the result image according to the setting after the colorcalibration, or a UI 1414 for guiding the user's selection for applyingthe setting after the color calibration. For example, if the userselects the UI 1412 for guiding to project the result image according tothe setting before the color calibration, the electronic apparatus 100may control the projection part 120 to project the result image 1410according to the setting before the color calibration.

Referring to FIG. 14, 1402 may indicate the electronic apparatus 100projecting a result image 1420 according to the setting after the colorcalibration. For example, if the user selects the UI 1413 for guiding toproject the result image according to the setting after the colorcalibration, the electronic apparatus 100 may control the projectionpart 120 to project the result image 1420 according to the setting afterthe color calibration.

The result image 1410 may be an image projected according to the settingbefore the color calibration. Accordingly, the result image 1410 may berecognized differently due to the color of the screen 10. However, theresult image 1420 may be an image projected according to the settingafter the color calibration. Accordingly, the result image 1420 may beexpressed in the color of the image as it is that was initiallyintended, in spite of the color of the screen 10.

The result image projected without a separate user selection after thecolor calibrating operation may be any one image between the resultimage 1410 of the embodiment 1401 and the result image 1420 of theembodiment 1402.

FIG. 15 is a diagram illustrating an example operation of comparingprojection before color calibration and projection after colorcalibration according to various embodiments.

Referring to FIG. 15, the electronic apparatus 100 may project a resultimage 1510. The result image 1510 may be divided into two areas. Thefirst area 1511 may be an area projected to the setting before the colorcalibration among the entire areas of the result image 1510. The secondarea 1512 may be an area projected to the setting after the colorcalibration among the entire areas of the result image 1510. Theelectronic apparatus 100 may project some areas in the result image 1510to the setting before the color calibration, and project the remainingareas to the setting after the color calibration.

FIG. 16 is a diagram illustrating an example operation of projectinginformation related to a terminal apparatus that can be connected withthe electronic apparatus 100 according to various embodiments.

Referring to FIG. 16, the electronic apparatus 100 may project an image1610 including a list of at least one apparatus that can be connectedwith the electronic apparatus 100. For example, it is assumed that theelectronic apparatus 100 is in a state of being able to be connectedwith the first terminal apparatus, the second terminal apparatus, andthe third terminal apparatus. The electronic apparatus 100 may projectinformation corresponding to the three terminal apparatuses as a list.If a UI 1611 for connecting with the first terminal apparatus isselected by the user, the electronic apparatus 100 may performconnection of communication with the first terminal apparatus.

When the electronic apparatus 100 is communicatively connected with aspecific terminal apparatus, the electronic apparatus 100 may project aresult image 1620. The result image 1620 may include at least one ofinformation 1621 notifying that the electronic apparatus 100 isconnected with a specific terminal apparatus or a UI 1622 for selectingwhether to maintain connection with the specific terminal apparatus. TheUI 1622 for selecting whether to maintain connection with the specificterminal apparatus may include at least one of a UI 1623 correspondingto maintenance of connection or a UI 1624 corresponding to release ofconnection. If the UI 1624 corresponding to release of connection isselected by the user, the electronic apparatus 100 may finish connectionof communication with the specific terminal apparatus previouslyconnected (e.g., the first terminal apparatus).

After connection of communication with the specific terminal apparatus(e.g., the first terminal apparatus) is finished, the electronicapparatus 100 may project an image 1630 including the list ofconnectable apparatuses again. The image 1630 may project the UIs 1631,1632 corresponding to apparatuses that can be connected with theelectronic apparatus 100 but of which connection was released in layoutsdifferent from the other UIs. For example, the electronic apparatus 100may display the UIs 1631, 1632 corresponding to the first terminalapparatus of which connection was already released in a dim color or ina gray color.

FIG. 17 is a flowchart illustrating an example process of performing anoperation of acquiring a color space transformation matrix and anoperation of performing a color calibrating operation according tovarious embodiments.

Referring to FIG. 17, the electronic apparatus 100 may acquire a firstphotographed image including the screen member 30 in operation S1705.The first photographed image may be photographed by the electronicapparatus 100 or the terminal apparatus 200. The electronic apparatus100 may acquire a color space transformation matrix in operation S1710.The color space transformation matrix may be generated based on thefirst photographed image. The electronic apparatus 100 may acquire asecond photographed image including the screen (e.g., projectionsurface) 10 in operation S1715. The second photographed image may bephotographed by the electronic apparatus 100 or the terminal apparatus200. The electronic apparatus 100 may perform a color calibratingoperation in operation S1720. The color calibrating operation may beperformed based on the second photographed image.

FIG. 18 is a flowchart illustrating an example operation of acquiring acolor space transformation matrix according to various embodiments.

Referring to FIG. 18, the electronic apparatus 100 may project a firstpattern image on the screen member 30 in operation S1805. The electronicapparatus 100 may acquire a first photographed image including thescreen member 30 on which the first pattern image is projected inoperation S1810. The electronic apparatus 100 may acquire a color spacetransformation matrix based on RGB information corresponding to thefirst photographed image and color space information corresponding tothe first pattern image in operation S1815. The electronic apparatus 100may project a second pattern image on the screen 10 in operation S1820.The electronic apparatus 100 may acquire a second photographed imageincluding the screen 10 on which the second pattern image is projectedin operation S1825. The electronic apparatus 100 may perform colorcalibration based on RGB information corresponding to the secondphotographed image, the color space transformation matrix, and colorspace information corresponding to the second pattern image in operationS1830.

FIG. 19 is a flowchart illustrating an example color calibratingoperation according to various embodiments.

Referring to FIG. 19, after acquiring a color space transformationmatrix, the electronic apparatus 100 may project a second pattern imageon the screen (e.g., projection surface) 10 in operation S1905. Theelectronic apparatus 100 may acquire a second photographed imageincluding the screen (e.g., projection surface) 10 on which the secondpattern image is projected in operation S1910. The electronic apparatus100 may transform (or convert) RGB information corresponding to thesecond photographed image into color space information corresponding tothe second photographed image based on the color space transformationmatrix in operation S1915. The electronic apparatus 100 may acquire acolor difference between the color space transformation corresponding tothe second photographed image and the color space informationcorresponding to the second pattern image in operation S1920.

The electronic apparatus 100 may identify whether the color differenceis greater than or equal to a threshold value in operation S1925. If thecolor difference is greater than or equal to the threshold value inoperation S1925—Y, the electronic apparatus 100 may perform colorcalibration based on the color difference in operation S1930. Afterperforming the color calibration, the electronic apparatus 100 mayproject the second pattern image again. The electronic apparatus 100 mayrepeat the operations S1905 to S1925. If the color difference is lessthan the threshold value in operation S1925—N, the electronic apparatus100 may maintain the currently set value without performing colorcalibration in operation S1935.

FIG. 20 is a flowchart illustrating an example operation of identifyingwhether a predetermined object is included in the screen member 30according to various embodiments.

Referring to FIG. 20, in the first embodiment and the third embodimentof FIG. 4 wherein a user directly installs the screen member 30, thereis a need to check whether the screen member 30 is a genuine screenmember 30 corresponding to the electronic apparatus 100. This isbecause, in case the screen member 30 is not a genuine screen member 30,color calibration may not be performed correctly.

The electronic apparatus 100 may acquire a first photographed imageincluding the screen member 30 in operation S2005. The electronicapparatus 100 may identify whether a predetermined (e.g., specified)object is included in the first photographed image in operation S2010.The predetermined object may refer, for example, to an object foridentifying whether the screen member 30 is a screen membercorresponding to the electronic apparatus 100. A more detailedexplanation in this regard will be provided below with reference to FIG.21 and FIG. 22.

If the predetermined object is included in the first photographed imagein operation S2010—Y, the electronic apparatus 100 may acquire a colorspace transformation matrix in operation S2015. The feature that thepredetermined object is included in the first photographed image mayrefer, for example, to the screen member 30 existing on the screen 10.The predetermined object may be an object related to the screen member30.

If the predetermined object is not included in the first photographedimage in operation S2010—N, the electronic apparatus 100 may project atext that the screen member 30 is not recognized in operation S2020. Thefeature that the predetermined object is not included in the firstphotographed image may refer, for example, to the screen member 30 notexisting on the screen 10. However, there may be a circumstance whereinthere is a different type of screen member which is not a genuine screenmember 30. In such a case, the electronic apparatus 100 may project atext notifying that there is a different type of screen member which isnot a genuine product.

FIG. 21 is a diagram illustrating an example operation of identifyingthe screen member 30 according to various embodiments.

Referring to FIG. 21, the screen member 30 may include a general area2111 and a bumpy area 2112. The general area 2111 may be an areaincluding a plane having a regular reflection rate. The bumpy area 2112may be an area including a surface of which reflection rate varies. Inthe general area 2111, the reflection rate of light is regular, but inthe bumpy area 2112, the reflection rate may not be regular. The bumpyarea 2112 may, for example, be an area manufactured with intaglio(engraving) or relief (embossing). The bumpy area 2112 may be apredetermined (e.g., specified) object. If the predetermined object (thebumpy area 2112) is included in the acquired first photographed image,the electronic apparatus 100 may determine that the screen member 30 isa genuine product.

In FIG. 21, the bumpy area 2112 is indicated with slashes, but in actualimplementation, the bumpy area 2112 may be an area that is notrecognized by a user's vision in a general situation, but may berecognized by a user's vision limited to a situation wherein light isirradiated. The bumpy area 2112 may be implemented as an uneven surface,and thus the reflection rate of light may not be regular.

FIG. 22 is a diagram illustrating an example operation of identifyingthe screen member 30 according to various embodiments.

Referring to FIG. 22, the screen member 30 may include a predetermined(e.g., specified) object 2210. The predetermined object 2210 may refer,for example, to an object that can indicate a genuine screen member. Forexample, the predetermined object 2210 may include, for example, atleast one of a predetermined text, a predetermined image, apredetermined icon, or a predetermined pattern. The predetermined object2210 may be an object that is not seen well in a state wherein light isnot irradiated, but that is seen well in a state wherein light isirradiated.

FIG. 23 is a signal flow diagram illustrating an example operation ofperforming a color calibrating in the electronic apparatus 100 accordingto various embodiments.

Referring to FIG. 23, the electronic apparatus 100 may perform colorcalibration by acquiring a color space transformation matrix. Forexample, the electronic apparatus 100 may project the first patternimage on the screen member 30 in operation S2305. The electronicapparatus 100 may transmit a first photographing command to the terminalapparatus 200 in operation S2310. In case the user directly performs aphotographing behavior after watching that the first pattern image isprojected, the operation S2310 may be omitted.

After the first pattern image is projected, the terminal apparatus 200may acquire a first photographed image including the screen member 30 onwhich the first pattern image is projected in operation S2315. Theterminal apparatus 200 may transmit the first photographed image to theelectronic apparatus 100 in operation S2320.

The electronic apparatus 100 may acquire a color space transformationmatrix based on the first photographed image received from the terminalapparatus 200 and the first pattern image in operation S2325. Forexample, the electronic apparatus 100 may acquire a color spacetransformation matrix based on RGB information corresponding to thefirst photographed image and color space information corresponding tothe first pattern image.

After acquiring the color space transformation matrix, the electronicapparatus 100 may project the second pattern image on the screen 10 inoperation S2330. After projecting the second pattern image, theelectronic apparatus 100 may transmit a second photographing command tothe terminal apparatus 200 in operation S2335. In case the user directlyperforms a photographing behavior after watching that the second patternimage is projected, the operation S2335 may be omitted.

After the second pattern image is projected, the terminal apparatus 200may acquire a second photographed image including the screen 10 on whichthe second pattern image is projected in operation S2340. The terminalapparatus 200 may transmit the second photographed image to theelectronic apparatus 100 in operation S2345.

The electronic apparatus 100 may perform color calibration based on thesecond photographed image received from the terminal apparatus 200, thecolor space transformation matrix, and the second pattern image inoperation S2350. For example, the electronic apparatus 100 may changeRGB information corresponding to the second photographed image to colorspace information corresponding to the second photographed image basedon the color space transformation matrix. The electronic apparatus 100may perform color calibration based on a color difference between thecolor space information corresponding to the second photographed imageand the color space information corresponding to the second patternimage.

FIG. 24 is a signal flow diagram illustrating an example operation ofperforming color calibration in the terminal apparatus 200 according tovarious embodiments.

Referring to FIG. 24, the electronic apparatus 100 may perform colorcalibration by acquiring a color space transformation matrix. Forexample, the electronic apparatus 100 may project the first patternimage on the screen member 30 in operation S2405. After projecting thefirst pattern image, the electronic apparatus 100 may transmit a firstphotographing command to the terminal apparatus 200 in operation S2410.In case the user directly performs a photographing behavior afterwatching that the first pattern image is projected, the operation S2410may be omitted.

After the first pattern image is projected, the terminal apparatus 200may acquire a first photographed image including the screen member 30 onwhich the first pattern image is projected in operation S2415.

The terminal apparatus 200 may acquire a color space transformationmatrix based on the first photographed image and the first pattern imagein operation S2420. For example, the terminal apparatus 200 may acquirea color space transformation matrix based on RGB informationcorresponding to the first photographed image and color spaceinformation corresponding to the first pattern image. The color spaceinformation corresponding to the first pattern image may be informationalready stored in the terminal apparatus 200, or information transmittedtogether in the operation S2410. The terminal apparatus 200 may transmita command for projecting the second pattern image to the electronicapparatus 100 in operation S2425.

If the command for projecting the second pattern image is received fromthe terminal apparatus 200, the electronic apparatus 100 may project thesecond pattern image on the screen 10 in operation S2430. Afterprojecting the second pattern image, the electronic apparatus 100 maytransmit a second photographing command to the terminal apparatus 200 inoperation S2435. In case the user directly performs a photographingbehavior after watching that the second pattern image is projected, theoperation S2435 may be omitted.

After the second pattern image is projected, the terminal apparatus 200may acquire a second photographed image including the screen 10 on whichthe second pattern image is projected in operation S2440.

The terminal apparatus 200 may perform color calibration based on thesecond photographed image, the color space transformation matrix, andthe second pattern image in operation 52445. For example, the terminalapparatus 200 may change RGB information corresponding to the secondphotographed image to color space information corresponding to thesecond photographed image based on the color space transformationmatrix. The terminal apparatus 200 may perform color calibration basedon a color difference between the color space information correspondingto the second photographed image and the color space informationcorresponding to the second pattern image. The color space informationcorresponding to the second pattern image may be information alreadystored in the terminal apparatus 200, or information transmittedtogether in the operation S2435. The terminal apparatus 200 may transmitthe color calibration result to the electronic apparatus 100 inoperation 52450.

Other than in FIG. 24, both of the operation of acquiring a color spacetransformation matrix and the color calibrating operation described tobe performed in the electronic apparatus 100 in the description of thedisclosure may be performed in the terminal apparatus 200.

FIG. 25 is a flowchart illustrating an example operation of performingcolor calibrating using streaming data according to various embodiments.

Referring to FIG. 25, the electronic apparatus 100 may acquire an imageincluding the screen member 30 or the screen 10 in real time. Forexample, the electronic apparatus 100 may acquire a real time streamingimage corresponding to a space wherein the electronic apparatus 100projects an image in operation 52505. The electronic apparatus 100 mayidentify whether the screen member 30 is included in the streaming imagein operation S2510.

If the screen member 30 is not included in the streaming image inoperation S2510—N, the electronic apparatus 100 may repeatedly acquire astreaming image. If the screen member 30 is included in the streamingimage in operation S2510—Y, the electronic apparatus 100 may project thefirst pattern image on the screen member 30 in operation S2515.

The electronic apparatus 100 may acquire a color space transformationmatrix based on the streaming image including the screen member 30 onwhich the first pattern image is projected in operation 52520. Forexample, the electronic apparatus 100 may acquire a color spacetransformation matrix based on RGB information corresponding to thestreaming image including the screen member 30 and color spaceinformation corresponding to the first pattern image.

After acquiring the color space transformation matrix, the electronicapparatus 100 may identify whether the screen member 30 is included inthe streaming image again in operation S2525.

If the screen member 30 is included in the streaming image afteracquiring the color space transformation matrix in operation S2525—Y,the electronic apparatus 100 may project a UI for guiding to remove thescreen member 30 in operation S2530. The electronic apparatus 100 mayrepeatedly identify whether the screen member 30 is included in thestreaming image. If the screen member 30 is not included in thestreaming image after acquiring the color space transformation matrix inoperation S2525—N, the electronic apparatus 100 may project the secondpattern image on the screen 10 in operation S2535.

The electronic apparatus 100 may perform color calibration based on thestreaming image including the screen 10 on which the second patternimage is projected in operation S2540. For example, the electronicapparatus 100 may obtain RGB information corresponding to the streamingimage including the screen 10 to color space information correspondingto the streaming image including the screen 10 based on the color spacetransformation matrix. The electronic apparatus 100 may perform colorcalibration based on a color difference between the color spaceinformation corresponding to the streaming image including the screen 10and the color space information corresponding to the second patternimage.

FIG. 26 is a diagram illustrating an example system including theelectronic apparatus 100, the terminal apparatus 200, and the server 300according to various embodiments.

Referring to FIG. 26, the server 300 may refer, for example, to anapparatus that can be communicatively connected with the electronicapparatus 100 and the terminal apparatus 200. The server 300 maytransmit information necessary for the electronic apparatus 100 or theterminal apparatus 200.

According to an embodiment, the electronic apparatus 100 and theterminal apparatus 200 may directly transmit and receive information.

According to an embodiment, the electronic apparatus 100 and theterminal apparatus 200 may transmit and receive information through theserver 300.

As an example, the terminal apparatus 200 may transmit a photographedimage to the server 300, and the server 300 may transmit the imagereceived from the terminal apparatus 200 to the electronic apparatus100.

As another example, color calibration may be performed in the terminalapparatus 200 using the color space transformation matrix. The server300 may transmit information related to the first pattern image andinformation related to the second pattern image to the terminalapparatus 200. The information related to the first pattern image mayrefer, for example, to color space information corresponding to thefirst pattern image. The information related to the second pattern imagemay refer, for example, to color space information corresponding to thesecond pattern image. The color calibration result generated by theterminal apparatus 200 may be transmitted to the server 300, and theserver 300 may transmit the color calibration result received from theterminal apparatus 200 to the electronic apparatus 100.

FIG. 27 is a diagram illustrating an example process of acquiring acolor space transformation matrix used in acquiring XYZ color spaceinformation corresponding to a photographed image according to variousembodiments.

Referring to FIG. 27, 2700 describes a schematic process of acquiring acolor space transformation matrix. The color space transformation matrixmay refer, for example, a matrix for converting from RGB informationinto XYZ color space information. The matrix 2701 may refer, forexample, to the color space transformation information or the colorspace transformation matrix. The matrix 2701 may refer, for example, toa 3*3 matrix.

The matrix 2701 may include nine unknowns. The nine unknowns may be KXR,KXG, KXB, KYR, KYG, KYB, KZR, KZG, and KZB. For acquiring correct valuesfor the nine unknowns, the electronic apparatus 100 may use the threeexamples 2710, 2720, 2730. 2710 is an example of projecting a redpattern image, and then photographing the image, 2720 is an example ofprojecting a green pattern image, and photographing the image, and 2730is an example of projecting a blue pattern image, and then photographingthe image.

In 2710, the electronic apparatus 100 may project a red pattern image,and acquire a photographed image including the screen member 30 on whichthe red pattern image is projected. The electronic apparatus 100 mayacquire RGB information corresponding to the red pattern image from theacquired photographed image. The matrix 2701 may refer, for example, toa color space transformation matrix, and it may refer, for example, to a3*3 matrix having nine unknowns. The matrix 2712 may refer, for example,to RGB information corresponding to the photographed image including thescreen member 30 on which the red pattern image is projected. The matrix2713 may refer, for example, to XYZ color space informationcorresponding to the red pattern image. The matrix 2713 may be stored inthe memory 110 in advance before acquiring the photographed image.Through 2710, three simultaneous equations related to the color spacetransformation matrix may be acquired.

In 2720, the electronic apparatus 100 may project a green pattern image,and acquire a photographed image including the screen member 30 on whichthe green pattern image is projected. The electronic apparatus 100 mayacquire RGB information corresponding to the green pattern image fromthe acquired photographed image. The matrix 2701 may refer, for example,to a color space transformation matrix, and it may refer, for example,to a 3*3 matrix having nine unknowns. The matrix 2722 may refer, forexample, to RGB information corresponding to the photographed imageincluding the screen member 30 on which the green pattern image isprojected. The matrix 2723 may refer, for example, to XYZ color spaceinformation corresponding to the green pattern image. The matrix 2723may be stored in the memory 110 in advance before acquiring thephotographed image. Through 2720, three simultaneous equations relatedto the color space transformation matrix may be acquired.

In 2730, the electronic apparatus 100 may project a blue pattern image,and acquire a photographed image including the screen member 30 on whichthe blue pattern image is projected. The electronic apparatus 100 mayacquire RGB information corresponding to the blue pattern image from theacquired photographed image. The matrix 2701 may refer, for example, toa color space transformation matrix, and it may refer, for example, to a3*3 matrix having nine unknowns. The matrix 2732 may refer, for example,to RGB information corresponding to the photographed image including thescreen member 30 on which the blue pattern image is projected. Thematrix 2733 may refer, for example, to XYZ color space informationcorresponding to the blue pattern image. The matrix 2733 may be storedin the memory 110 in advance before acquiring the photographed image.Through 2730, three simultaneous equations related to the color spacetransformation matrix may be acquired.

The electronic apparatus 100 may acquire nine simultaneous equationsthrough the three kinds of embodiments 2710, 2720, 2730, and find all ofthe nine unknowns included in the color space transformation matrixusing the nine simultaneous equations. Ultimately, the electronicapparatus 100 may acquire a color space transformation matrix of a 3*3matrix.

In FIG. 27, a 3*3 matrix was acquired for acquiring a color spacetransformation matrix corresponding to the embodiment of using XYZ colorspace information in RGB information. However, depending onimplementation examples, various kinds of color space information may beused other than XYZ color space information. Depending on implementationexamples, the electronic apparatus 100 may acquire a color spacetransformation matrix in a different size other than a 3*3 matrix.

FIG. 28 is a diagram illustrating RGB information corresponding to aphotographed image and XYZ color space information corresponding to aphotographed image according to various embodiments.

Referring to FIG. 28, the electronic apparatus 100 may transform RGBinformation corresponding to a photographed image into XYZ color spaceinformation. For example, the electronic apparatus 100 may acquire XYZcolor space information by multiplying RGB information corresponding toa photographed image with a color space transformation matrix. The table2810 may indicate the RGB information, and the table 2820 may indicatethe color space transformation information.

FIG. 29 is a flowchart illustrating an example method of controlling anelectronic apparatus according to various embodiments.

A method of controlling the electronic apparatus 100 that can store afirst pattern image and a second pattern image and that communicateswith an external terminal apparatus according to an example embodimentof the disclosure includes: projecting a first pattern image on a screenmember 30 located on a projection surface (S2905), and based onreceiving a first photographing image which photographed the screenmember 30 from the external terminal apparatus, acquiring transformationinformation based on the first photographed image and the first patternimage (S2910), projecting a second pattern image on the projectionsurface (S2915), and based on receiving a second photographed imagewhich photographed the projection surface from the external terminalapparatus, performing color calibration according to the characteristicof the projection surface based on the second photographed image, thesecond pattern image, and the transformation information (S2920).

The transformation information may be color space transformationinformation, and in the step of acquiring the transformation information(S2910), color space information corresponding to the first patternimage may be acquired, and the color space transformation informationaccording to the characteristic of a projection part 120 included in theelectronic apparatus 100 may be acquired based on the first photographedimage and the color space information corresponding to the first patternimage.

In the operation of performing the color calibration (S2920), colorspace information corresponding to the second pattern image may beacquired, and color calibration according to the characteristic of theprojection surface may be performed based on the second photographedimage, the color space information corresponding to the second patternimage, and the color space transformation information.

In the operation of acquiring the transformation information (S2910),the color space transformation information may be acquired based on RGBinformation corresponding to the first photographed image and XYZ colorspace information corresponding to the first pattern image.

The transformation information may be a color space transformationmatrix transforming the RGB information into the XYZ color spaceinformation.

In the operation of performing the color calibration (S2920), the RGBinformation corresponding to the second photographed image may betransformed into the XYZ color space information based on thetransformation information, a color difference between the XYZ colorspace information corresponding to the second photographed image and theXYZ color space information corresponding to the second pattern imagemay be acquired, and the color calibration may be performed based on theacquired color difference.

In the operation of performing the color calibration (S2920), at leastone of a gain value or an offset value related to an RGB signal may beacquired based on the acquired color difference.

The method may further include: based on identifying that apredetermined object related to the screen member 30 is included in thefirst photographed image, acquiring the transformation information basedon the first photographed image and the first pattern image, and basedon identifying that a specified object related to the screen member 30is not included in the first photographed image, projecting a UIincluding information that the screen member 30 is not recognized.

The first pattern image may include at least one of a white patternimage, a red pattern image, a green pattern image, or a blue patternimage, and the second pattern image may include a white pattern image.

In the operation of projecting the first pattern image (S2905), thewhite pattern image among the plurality of pattern images included inthe first pattern image may be projected first, and the remainingpattern images may be sequentially projected.

The first pattern image may include at least one of a red pattern, agreen pattern, a blue pattern, or a white pattern. Here, the electronicapparatus 100 may output the white pattern first. The electronicapparatus 100 may output the red, green, and blue patterns. If the whitepattern is output first, the order of the red, green, and blue patternsafter than may vary according to the user setting.

The screen member 30 may, for example, be a white screen member. As anexample, the screen member 30 may be a material such as plastic, etc.,and it may be a flat material that can be attached to or mounted on theprojection surface as describe by way of example above with reference toFIG. 5 and FIG. 7. As another example, a part of the electronicapparatus 100 may be designed to be utilized as the screen member. Thescreen member 30 may be a mechanical part or a panel, and it may be apartial component of the electronic apparatus 100 as described by way ofexample above with reference to FIG. 6, FIG. 8, and FIG. 10.

The rim of the screen member 30 may have been designed in a patternhaving a reflection rate that is different as much as the thicknessand/or the depth defined by the manufacturer. Accordingly, when aspecific light is projected on the screen member 30 from the projectionpart, the reflection rate of the rim and the reflection rates of theother areas may be different. Accordingly, in case the screen member 30is photographed by the camera, the gradation (the brightness or thecolor) may be recognized differently depending on the areas of thescreen member 30.

The pattern having a different reflection rate may be designed such thatit is recognized well in case a light having directivity is projected asthe electronic apparatus 100, but is difficult to be distinguished undera general natural light or an indoor lamp. By such a method, a specificshape or letter may be imprinted, and may be recognized whenphotographed by the camera.

A user may position the screen member 30 provided when purchasing theelectronic apparatus 100 in a location that an output image isprojected, and positions the screen member 30 such that an entire outputimage or a portion of an output image is projected on the screen member30. For identifying the screen member defined by the manufacturer, theelectronic apparatus 100 may identify with the naked eye by projecting apattern or a light defined in advance, or for automation, the electronicapparatus 100 may provide a calibration function such that, if thescreen member is photographed by the camera provided on the electronicapparatus 100 or a smartphone camera, the calibration function isoperated when the screen member is recognized.

According to an embodiment, a pre-calibrating operation may be performedas a menu provided at the electronic apparatus 100 is selected. Thepre-calibrating operation may include operations of communicativelyconnecting the electronic apparatus 100 projecting an image and aterminal apparatus 200, and adjusting the sensitivity of the projectionpart 120.

If a pre-calibration menu among SW OSD menus of the electronic apparatus100 is selected, the electronic apparatus 100 may find a user terminalapparatus 200 on which a camera is mounted in the surroundings, andoutput a list. If a user selects a specific device, the electronicapparatus 100 may output a message attempting connection to the specificdevice, and wait for acceptance (or selection) of the user. Theelectronic apparatus 100 may automatically find surrounding devicesthrough methods such as WiFi, UPNP, etc., and output a list. Here, thelist may include at least one of device names, device IDs, or deviceshapes that iconized schematic shapes of devices. When a specific deviceis selected by the user, the electronic apparatus 100 may set paringconnection temporarily, and output an image indicating which is thedevice that the user currently selected (for which the color calibratingfunction will be performed).

If the device is not the device wanted by the user, the electronicapparatus 100 may release the temporary connection and may output thelist again, and may output a guide image so that the user can select theconnected device again. The electronic apparatus 100 may outputinformation related to the device of which connection was released in agray color, so that the device of which connection was released is notselected again. By displaying surrounding devices by distinguishingcolors, the user can easily distinguish and display surrounding devicesthat can be connected with the electronic apparatus 100.

When the user selects one device, one between the electronic apparatus100 and the selected device may become an access point (AP) and paringmay be operated, and it may become a state wherein wirelesscommunication between the electronic apparatus 100 and the selecteddevice is possible. The electronic apparatus 100 may store the name orthe physical address, etc. of the paired device, and may use the storedinformation again when pairing is needed again later. As connectioninformation is stored, the electronic apparatus 100 may provide anautomatic connecting function that can shorten the time for the user toselect.

According to an embodiment, the pre-color calibrating function may beperformed as a menu provided at the terminal apparatus 200 is selected.

If a pre-calibration menu provided at an application of the terminalapparatus 200 is selected, the terminal apparatus 200 may findsurrounding devices that can be communicatively connected with theterminal apparatus 200, and display subjects having a projectionfunction in the terminal apparatus 200 as a list. The user may select aspecific device in the list displayed through the terminal apparatus200. The list may include at least one of device names, device IDs, orthumbnails of images that are being reproduced (or output) in thesurrounding devices.

If one surrounding device is selected in the list, one between theterminal apparatus 200 and the selected device may become an accesspoint (AP) and paring may be operated, and it may become a state whereinwireless communication between the terminal apparatus 200 and theselected device is possible. Here, the terminal apparatus 200 may storethe name or the physical address, etc. of the paired device, and may usethe stored information again when pairing is needed again later. Asconnection information is stored, the terminal apparatus 200 may providean automatic connecting function that can shorten the time for the userto select.

The pre-calibration may be a process of matching the display and thecalibration reference point for resolving the problems of calibrationprecision and distribution due to procedural distribution of variouscell phone cameras. Through this, the problem of procedural distributionof various smartphone cameras can be resolved.

An embodiment wherein a color calibrating operation is performed in theterminal apparatus 200 is assumed. The electronic apparatus 100 maychange the patterns of red, green, and blue and output the patterns onthe screen member 30, and transmit a signal (flag) for photographing animage to the terminal apparatus 200. Also, the terminal apparatus 200may analyze the photographed image, and transmit picture qualityadjustment information according to the analysis result and a signal(flag) for changing to the next pattern to the electronic apparatus 100.After the terminal apparatus 200 photographed the red pattern output bythe electronic apparatus 100, the red, green, and blue wavelengthinformation may be extracted from the acquired image and stored.Likewise, after the terminal apparatus 200 photographed the green andblue patterns output by the electronic apparatus 100, the respectivered, green, and blue wavelength information may be analyzed from theacquired image and stored. The white pattern may be analyzed and stored.By combining the pattern photographing analysis information and theR/G/B reaction characteristics of the camera included in the terminalapparatus 200, the terminal apparatus 200 may generate a color spacetransformation matrix for precise RGB to XYZ color domaintransformation. The color space transformation matrix may be used incolor space transformation for optical color coordinate adjustment fromthe RGB information of the image photographed by the camera of theterminal apparatus 200.

Even if the actual color of the screen is different, the electronicapparatus 100 or the terminal apparatus 200 may automatically calibratethe color to the picture quality of the original image or the picturequality set by the user.

If communicative connection is set between the terminal apparatus 200and the electronic apparatus 100, the color calibrating function may beperformed immediately without a need to select a device separately.

In case only a color space transformation matrix was generated in theprevious process, and the color calibrating operation was not completed,an operation for connecting the electronic apparatus 100 and theterminal apparatus 200 may be performed. If a specific menu is selectedby the user, a list may be output through the electronic apparatus 100,or a list may be output through an application of the terminal apparatus200.

If a specific menu is selected at the electronic apparatus 100 or theterminal apparatus 200 after communicative connection is set, theelectronic apparatus 100 may perform the color calibrating function.

In case the electronic apparatus 100 utilizes a wall surface having aspecific color as the screen 10, the color calibrating function may beneeded. Specifically, the electronic apparatus 100 may project the whitepattern on the screen member 30, and when projection is completed, theelectronic apparatus 100 may give a flag, and transmit a signal forphotographing to the terminal apparatus 200. After the terminalapparatus 200 photographed the white pattern output from the electronicapparatus 100 and reflected on the wall surface, the color calibratingfunction may be performed by utilizing only the information of apre-defined operation area (e.g., N×N pixels in the center of the image)among the entire areas of the RGB image.

The electronic apparatus 100 may transform the RGB information of thephotographed image into XYZ color space information by utilizing thecolor space transformation matrix, and then calculate a colorcoordinate. Here, it may be analyzed how much difference the calculatedcolor coordinate has from the target color coordinate to be adjustedthrough the color calibrating function. The electronic apparatus 100 maycalculate a signal change (adjustment) value for calibration of thecolor coordinate, and inverse-transform the color coordinate calibrationvalue in this XYZ color space into the RGB color space again (it can becalculated as an inverse-transformation matrix of the transformationmatrix), and calculate the RGB calibration value. The electronicapparatus 100 may adjust the output signal level by utilizing a gainand/or an offset for the RGB signal. At the time point when an errorbecomes smaller than or equal to a predetermined threshold value byrepetitively performing this process, the calibration process may becompleted.

An embodiment wherein a color calibrating operation is performed in theterminal apparatus 200 is assumed. The electronic apparatus 100 mayoutput the white pattern, and give a flag for photographing to theterminal apparatus 200. When the terminal apparatus 200 completesphotographing and analysis, the terminal apparatus 200 may transmit theresult value that it analyzed and a flag for changing to the nextpattern to the electronic apparatus 100.

An embodiment wherein a color calibrating operation is performed in theelectronic apparatus 100 is assumed. The camera of the terminalapparatus 200 may transmit photographed RGB information to theelectronic apparatus 100, and the electronic apparatus 100 may performall of color space transformation, calculation of an RGB signal outputadjustment value, and adjustment. For this, the color spacetransformation matrix acquired in the pre-calibration step may be storedin the electronic apparatus 100, and it may then be shared to theterminal apparatus 200, or vice versa. The color space transformationmatrix may be kept in the server 300, and may be downloaded immediatelyon a needing side and used.

In the case of performing the color calibrating function, the electronicapparatus 100 may calculate only the operation area of the RGB image(the central N×N pixels of the RGB image) acquired by the camera of theterminal apparatus 200, and perform color calibration based on thecentral area, for example. As another example, the electronic apparatus100 may divide the acquired RGB image into rectangular areas in X and Ynumbers respectively in horizontal and vertical directions, andrespectively extract n×n pixels in the centers of the XxY rectangularareas. The electronic apparatus 100 may transform these into XYZ colorspaces respectively, and compare the color coordinates in the XxY numberand the color coordinate of the central area of the RGB image, andcalibrate color non-uniformity on the wall surface at the same time. Forexample, the electronic apparatus 100 may calculate the deviationbetween the color coordinate of the central area and the colorcoordinates of the surrounding areas, and calculate the non-uniformitycharacteristic of the projected wall surface. When performing signaladjustment in the electronic apparatus 100, the electronic apparatus 100may perform calibration through a process of mis-adjusting so thatuniformity of the RGB signal can be matched as it proceeds more to theoutside based on the center of the wall surface. When uniform opticalsignal information projected from the electronic apparatus 100 isreflected on the wall surface that the user is going to project andrecognized by the eyes of the user, the color non-uniformity of the wallsurface can be adjusted precisely.

The performing of an operation process by the terminal apparatus 200 orby the electronic apparatus 100 as above may be changed to a method ofcompletely changing the roles of each other automatically or by dividingsome of the process and processing them, etc. under the determination bythe terminal apparatus 200 or the electronic apparatus 100 inconsideration of the processing power between the two apparatuses. Also,such distribution of roles may be implemented in a form of beingselected by the user, so that it may be fixed as an option or changedflexibly.

In a process of initially photographing and analyzing, the electronicapparatus 100 may output a result image indicating to which level thewall surface can be calibrated precisely by the wall color calibratingfunction aimed to be provided in the disclosure. Here, the result imagemay include a UI notifying a processing result of the logic inside theproduct and the pre-determined rules. The electronic apparatus 100 maymeasure the color of the wall surface, and project the white pattern onthe wall surface of the current level. Then, the electronic apparatus100 may analyze the photographing result and calculate a signal changerange (e.g., min, max) that can be adjusted at the electronic apparatus100, and calculate how precisely the wall surface can be adjusted. Incase the precision is lower than the minimum level that is aimed to beprovided by the disclosure according to the measurement result of thesignal processing, the electronic apparatus 100 may output at least oneof a warning image, or a guide for the precision. The electronicapparatus 100 may provide an operation of identifying whether the userwill proceed with calibration or stop calibration after providing theguide for the predicted precision of calibration.

Even if the color of the projection surface has a specific color whichis not a white color through this process, precise adjustment to thecolor coordinate of the intended XYZ color space is possible. Althoughcalibration of the color on the previous RGB color space is differentfrom the recognized color, adjustment wherein the color seems to be thesame color as that of the original image such as a white screen ispossible through the method of calibrating the color by utilizing theXYZ space according to the disclosure. After adjustment is completed,the electronic apparatus 100 may display the change of the adjustedpicture quality before the adjustment and after the adjustment. Then,the electronic apparatus 100 may provide a UI so that the user canselect the setting before the adjustment or after the adjustment.

If the setting is finally selected by the user, the electronic apparatus100 may store the setting value. In case the electronic apparatus 100 isused on the same wall surface later, the electronic apparatus 100 maycall the stored setting value again and use it. The setting value may beimplemented in a form of being stored in the terminal apparatus 200 orthe server 300 other than the electronic apparatus 100.

The electronic apparatus 100 may store information on a wall surface ofwhich picture quality was adjusted (information that can specify thewall surface in an adjustment process such as the location in a home,the measured color distribution when projecting a white pattern on thewall surface from a projector, etc.), and easily apply the informationin a situation wherein a user uses a portable projector which isconvenient to carry.

The electronic apparatus 100 can provide the picture quality of anoriginal image without a change regardless of which material or a colorof a screen. Accordingly, functionality or usability of a projector canbe improved.

While the disclosure has been illustrated and described with referenceto various example embodiments, it will be understood that the variousexample embodiments are intended to be illustrative, not limiting. Itwill be further understood by those skilled in the art that variouschanges in form and detail may be made without departing from the truespirit and full scope of the disclosure, including the appended claimsand their equivalents. It will also be understood that any of theembodiment(s) described herein may be used in conjunction with any otherembodiment(s) described herein.

What is claimed is:
 1. An electronic apparatus comprising: a memorystoring a first pattern image and a second pattern image; acommunication interface comprising communication circuitry configured tocommunicate with an external terminal apparatus; a projection part; anda processor configured to: control the projection part to project thefirst pattern image on a screen member located on a projection surface,based on receiving a first photographed image which photographed thescreen member from the external terminal apparatus through thecommunication interface, acquire transformation information based on thefirst photographed image and the first pattern image, control theprojection part to project the second pattern image on the projectionsurface, and based on receiving a second photographed image whichphotographed the projection surface from the external terminal apparatusthrough the communication interface, perform color calibration accordingto the characteristic of the projection surface based on the secondphotographed image, the second pattern image, and the transformationinformation.
 2. The electronic apparatus of claim 1, wherein thetransformation information comprises color space transformationinformation, and the processor is configured to: acquire color spaceinformation corresponding to the first pattern image, and acquire thecolor space transformation information based on the characteristic ofthe projection part based on the first photographed image and the colorspace information corresponding to the first pattern image.
 3. Theelectronic apparatus of claim 2, wherein the processor is configured to:acquire color space information corresponding to the second patternimage, and perform color calibration based on the characteristic of theprojection surface based on the second photographed image, the colorspace information corresponding to the second pattern image, and thecolor space transformation information.
 4. The electronic apparatus ofclaim 2, wherein the processor is configured to: acquire the color spacetransformation information based on red, green blue (RGB) informationcorresponding to the first photographed image and XYZ color spaceinformation corresponding to the first pattern image.
 5. The electronicapparatus of claim 1, wherein the transformation information comprises acolor space transformation matrix transforming the RGB information intothe XYZ color space information.
 6. The electronic apparatus of claim 1,wherein the processor is configured to: transform the RGB informationcorresponding to the second photographed image into the XYZ color spaceinformation based on the transformation information, acquire a colordifference between the XYZ color space information corresponding to thesecond photographed image and the XYZ color space informationcorresponding to the second pattern image, and perform the colorcalibration based on the acquired color difference.
 7. The electronicapparatus of claim 6, wherein the processor is configured to: change atleast one of a gain value or an offset value related to an RGB signalbased on the acquired color difference.
 8. The electronic apparatus ofclaim 1, wherein the processor is configured to: based on identifyingthat a predetermined object related to the screen member is included inthe first photographed image, acquire the transformation informationbased on the first photographed image, and based on identifying that apredetermined object related to the screen member is not included in thefirst photographed image, control the projection part to project a userinterface (UI) including information that the screen member is notrecognized.
 9. The electronic apparatus of claim 1, wherein the firstpattern image includes at least one of a white pattern image, a redpattern image, a green pattern image, or a blue pattern image, and thesecond pattern image includes a white pattern image.
 10. The electronicapparatus of claim 9, wherein the processor is configured to: controlthe projection part to project the white pattern image among theplurality of pattern images included in the first pattern image first,and to project the remaining pattern images sequentially.
 11. A methodof controlling an electronic apparatus storing a first pattern image anda second pattern image and communicating with an external terminalapparatus, the method comprising: projecting the first pattern image ona screen member located on a projection surface; based on receiving afirst photographed image which photographed the screen member from theexternal terminal apparatus, acquiring transformation information basedon the first photographed image and the first pattern image; projectingthe second pattern image on the projection surface; and based onreceiving a second photographed image which photographed the projectionsurface from the external terminal apparatus, performing colorcalibration based on the characteristic of the projection surface basedon the second photographed image, the second pattern image, and thetransformation information.
 12. The method of claim 11, wherein thetransformation information comprises color space transformationinformation, and the acquiring the transformation information comprises:acquiring color space information corresponding to the first patternimage; and acquiring the color space transformation information based onthe characteristic of a projection part included in the electronicapparatus based on the first photographed image and the color spaceinformation corresponding to the first pattern image.
 13. The method ofclaim 12, wherein the performing the color calibration comprises:acquiring color space information corresponding to the second patternimage; and performing color calibration based on the characteristic ofthe projection surface based on the second photographed image, the colorspace information corresponding to the second pattern image, and thecolor space transformation information.
 14. The method of claim 12,wherein the acquiring the transformation information comprises:acquiring the color space transformation information based on red greenblue (RGB) information corresponding to the first photographed image andXYZ color space information corresponding to the first pattern image.15. The method of claim 11, wherein the transformation information is acolor space transformation matrix transforming the RGB information intothe XYZ color space information.
 16. The method of claim 11, wherein theperforming the color calibration comprises: transforming the RGBinformation corresponding to the second photographed image into the XYZcolor space information based on the transformation information,acquiring a color difference between the XYZ color space informationcorresponding to the second photographed image and the XYZ color spaceinformation corresponding to the second pattern image, and performingthe color calibration based on the acquired color difference.
 17. Themethod of claim 16, wherein the performing the color calibrationcomprises: changing at least one of a gain value or an offset valuerelated to an RGB signal based on the acquired color difference.
 18. Themethod of claim 11, further comprising: based on identifying that apredetermined object related to the screen member is included in thefirst photographed image, acquiring the transformation information basedon the first photographed image, and based on identifying that apredetermined object related to the screen member is not included in thefirst photographed image, projecting a user interface (UI) includinginformation that the screen member is not recognized.
 19. The method ofclaim 11, wherein the first pattern image includes at least one of awhite pattern image, a red pattern image, a green pattern image, or ablue pattern image, and the second pattern image includes a whitepattern image.
 20. The method of claim 19, wherein the projecting thefirst pattern image comprises: projecting the white pattern image amongthe plurality of pattern images included in the first pattern imagefirst, and projecting the remaining pattern images sequentially.